Machine Learning Techniques for Personalized Detection of Epileptic Events in Clinical Video Recordings

Continuous patient monitoring is essential to achieve an effective and optimal patient treatment in the intensive care unit. In the specific case of epilepsy it is the only way to achieve a correct diagnosis and a subsequent optimal medication plan if possible. In addition to automatic vital sign monitoring, epilepsy patients need manual monitoring by trained personnel, a task that is very difficult to be performed continuously for each patient. Moreover, epileptic manifestations are highly personalized even within the same type of epilepsy. In this work we assess two machine learning methods, dictionary learning and an autoencoder based on long short-term memory (LSTM) cells, on the task of personalized epileptic event detection in videos, with a set of features that were specifically developed with an emphasis on high motion sensitivity. According to the strengths of each method we have selected different types of epilepsy, one with convulsive behaviour and one with very subtle motion. The results on five clinical patients show a highly promising ability of both methods to detect the epileptic events as anomalies deviating from the stable/normal patient status

Wize Mirror - a smart, multisensory cardio-metabolic risk monitoring system

In the recent years personal health monitoring systems have been gaining popularity, both as a result of the pull from the general population, keen to improve well-being and early detection of possibly serious health conditions and the push from the industry eager to translate the current significant progress in computer vision and machine learning into commercial products. One of such systems is the Wize Mirror, built as a result of the FP7 funded SEMEOTICONS (SEMEiotic Oriented Technology for Individuals CardiOmetabolic risk self-assessmeNt and Self-monitoring) project. The project aims to translate the semeiotic code of the human face into computational descriptors and measures, automatically extracted from videos, multispectral images, and 3D scans of the face. The multisensory platform, being developed as the result of that project, in the form of a smart mirror, looks for signs related to cardio-metabolic risks. The goal is to enable users to self-monitor their well-being status over time and improve their life-style via tailored user guidance. This paper is focused on the description of the part of that system, utilising computer vision and machine learning techniques to perform 3D morphological analysis of the face and recognition of psycho-somatic status both linked with cardio-metabolic risks. The paper describes the concepts, methods and the developed implementations as well as reports on the results obtained on both real and synthetic datasets

Mirror mirror on the wall... an unobtrusive intelligent multisensory mirror for well-being status self-assessment and visualization

A person’s well-being status is reflected by their face through a combination of facial expressions and physical signs. The SEMEOTICONS project translates the semeiotic code of the human face into measurements and computational descriptors that are automatically extracted from images, videos and 3D scans of the face. SEMEOTICONS developed a multisensory platform in the form of a smart mirror to identify signs related to cardio-metabolic risk. The aim was to enable users to self-monitor their well-being status over time and guide them to improve their lifestyle. Significant scientific and technological challenges have been addressed to build the multisensory mirror, from touchless data acquisition, to real-time processing and integration of multimodal data

Automatic Assessment of Depression Based on Visual Cues: A Systematic Review

International audienceAutomatic depression assessment based on visual cues is a rapidly growing research domain. The present exhaustive review of existing approaches as reported in over sixty publications during the last ten years focuses on image processing and machine learning algorithms. Visual manifestations of depression, various procedures used for data collection, and existing datasets are summarized. The review outlines methods and algorithms for visual feature extraction, dimensionality reduction, decision methods for classification and regression approaches, as well as different fusion strategies. A quantitative meta-analysis of reported results, relying on performance metrics robust to chance, is included, identifying general trends and key unresolved issues to be considered in future studies of automatic depression assessment utilizing visual cues alone or in combination with vocal or verbal cues.Visualization; Affective computing; Monitoring; Europe; Mood; Reliability; Tools; Depression assessment; affective computing; facial expression; machine learning; facial image analysi

Automated Characterization of Mouth Activity for Stress and Anxiety Assessment

International audienceNon-verbal information portrayed by human facial expression, apart from emotional cues also encompasses information relevant to psychophysical status. Mouth activities in particular have been found to correlate with signs of several conditions; depressed people smile less, while those in fatigue yawn more. In this paper, we present a semi-automated, robust and efficient algorithm for extracting mouth activity from video recordings based on Eigen-features and template-matching. The algorithm was evaluated for mouth openings and mouth deformations, on a minimum specification dataset of 640x480 resolution and 15 fps. The extracted features were the signals of mouth expansion (openness estimation) and correlation (deformation estimation). The achieved classification accuracy reached 89.17%. A second series of experimental results, for the preliminary evaluation of the proposed algorithm in assessing stress/anxiety, took place using an additional dataset. The proposed algorithm showed consistent performance across both datasets, which indicates high robustness. Furthermore, normalized openings per minute, and average openness intensity were extracted as video-based features, resulting in a significant difference between video recordings of stressed/anxious versus relaxed subjects

Détection de la dépression par l’analyse de la géométrie faciale et de la parole

International audienceDepression is one of the most prevalent mental disorders, burdening many people world-wide. A system with the potential of serving as a decision support system is proposed, based on novel features extracted from facial expression geometry and speech, by interpreting non-verbal manifestations of depression. The proposed system has been tested both in gender independent and gender based modes, and with different fusion methods. The algorithms were evaluated for several combinations of parameters and classification schemes, on the dataset provided by the Audio/Visual Emotion Challenge of 2013 and 2014. The proposed framework achieved a precision of 94.8% for detecting persons achieving high scores on a self-report scale of depressive symptomatology. Optimal system performance was obtained using a nearest neighbour classifier on the decision fusion of geometrical features in the gender independent mode, and audio based features in the gender based mode; single visual and audio decisions were combined with the OR binary operation.Les troubles d'humeur affectent de nombreuses personnes, la dépression étant la plus répandue. Les méthodes avec la prospective d'aide aux cliniciens dans le diagnostic sont proposées ici, en fonction de la géométrie de l'expression du visage et de la parole. Les approches indépendantes du genre et dépendantes du genre ont été testées, pour différentes combinaisons de caractéristiques visuelles et audio. L'évaluation et la quantification des méthodes développées, pour plusieurs ensembles de paramètres, sont effectuées sur l'ensemble de données fournies par le challenge Emotion Audio / Visual. Un score de F1 de 71.3% a été atteint pour détecter les individus signalant des scores élevés sur le BDI-II. La meilleure configuration du système comprenait un classificateur d'analyse discriminant pour les caractéristiques géométriques dans l'approche indépendante du genre. Facial Geometry and Speech Analysis for Depression Detection (PDF Download Available). Available from: https://www.researchgate.net/publication/317102616_Facial_Geometry_and_Speech_Analysis_for_Depression_Detection [accessed Sep 15, 2017]

Quantitative comparison of motion history image variants for video-based depression assessment

Abstract Depression is the most prevalent mood disorder and a leading cause of disability worldwide. Automated video-based analyses may afford objective measures to support clinical judgments. In the present paper, categorical depression assessment is addressed by proposing a novel variant of the Motion History Image (MHI) which considers Gabor-inhibited filtered data instead of the original image. Classification results obtained with this method on the AVEC’14 dataset are compared to those derived using (a) an earlier MHI variant, the Landmark Motion History Image (LMHI), and (b) the original MHI. The different motion representations were tested in several combinations of appearance-based descriptors, as well as with the use of convolutional neural networks. The F1 score of 87.4% achieved in the proposed work outperformed previously reported approaches

Depression Assessment by Fusing High and Low Level Features from Audio, Video, and Text

International audienceDepression is a major cause of disability world-wide. The present paper reports on the results of our participation to the depression sub-challenge of the sixth Audio/Visual Emotion Challenge (AVEC 2016), which was designed to compare feature modalities ( audio, visual, interview transcript-based) in gender-based and gender-independent modes using a variety of classification algorithms. In our approach, both high and low level features were assessed in each modality. Audio features were extracted from the low-level descriptors provided by the challenge organizers. Several visual features were extracted and assessed including dynamic characteristics of facial elements (using Landmark Motion History Histograms and Landmark Motion Magnitude), global head motion, and eye blinks. These features were combined with statistically derived features from pre-extracted features ( emotions, action units, gaze, and pose). Both speech rate and word-level semantic content were also evaluated. Classification results are reported using four different classification schemes: i) gender-based models for each individual modality, ii) the feature fusion model, ii) the decision fusion model, and iv) the posterior probability classification model. Proposed approaches outperforming the reference classification accuracy include the one utilizing statistical descriptors of low-level audio features. This approach achieved f1-scores of 0.59 for identifying depressed and 0.87 for identifying notdepressed individuals on the development set and 0.52/0.81, respectively for the test set

Fast FPGA prototyping for real-time image processing with very high-level synthesis

International audienc